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j15 To all whom it may concern: 

Be it known that we, Katayoon (NMI) Atefi and Ralph James Foley, both citizens of 
Canada and currently residing in the provinces of Ontario and Quebec, respectively, in Canada, 
have invented new and useful improvements in a 

METHOD AND SYSTEM FOR ASSESSING INFORMATION 
20 TECHNOLOGY SERVICE DELIVERY 



of which the following is a SPECIFICATION: 
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METHOD AND SYSTEM FOR ASSESSING INFORMATION 
TECHNOLOGY SERVICE DELTV^RY 

Cross Reference to Related Patents 

The present invention is related to the following documents, all of which are 
5 assigned to the assignee of the present invention and which are specifically incorporated herein 
by reference: 

Patent Application Serial No. 09/625,108 (docket END9-20DO-0026US1) filed July 25, ' 
2000 by Steven D. Barnes et al. and entitled "Method and System for System Integration", a 
document which is sometimes referred to in this document as the SI Patent. 
10 Patent Application Serial Na 09/506,260 (docket RAL9-99.0166) filed February 17, 

2000 by Brenda Barth-Harvey et al. and entitled "Automated Management Process for Enterprise 
Resource Planning System Implementation", a document which is sometimes referred to in this 
document as the Management Process Patent. 
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Background of the Invention 

Field of the Invention 

The present invention relates to techniques for assessing delivery of information 
technology (IT) services to customers. More particularly, the present invention includes an 
5 improved system and method for evaluating the maturity of the IT service delivery system and 
for identifying factors which cause or contribute to a customer's perception of poor service. The 
present invention includes a IT service matrix which matrix rates the service in several key 
attributes according to a level of maturity and increasing customer satisfaction, . 

Ki Background Art 

ifilO The description of the present invention which follows is based on a presupposition that 

f 11 the reader has a basic knov^edge of information technology (sometimes referred to as (kta 

processing) and the delivery of IT services to customers, whether the customers are members of 
^ ■ j the same corporate organization as the IT or^nization (in a captive IT facility) or in an 
outsourcing organization which is hired to provide IT services to a corporate organization 
15 (sometimes the IT function is partially outsourced and partially performed in-house). Such IT 
organizations frequently involve a significant cost to the corporation. This significant cost 
sometimes raises the question of whether the IT organization is worth the expense and whether it 
can be improved, which often entails a consulting engagement by an outside organization to 
review the IT services, identify weaknesses and propose changes which will deliver better IT 
20 services or reduce IT costs or some combination of the two. 
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One approach to evaluating the effectiveness of an IT operation is to focus on the 
processes and procedures which are employed by the IT operation itself This is sometimes 
referred to as a production method, since it focuses on the production of IT deliverables in 
assessing the effectiveness and the adequacy of the IT opeig-tion. This is also an inwardly aligned 
5 evaluation does assist in evaluating the efficiency of the IT infrastructure by aligning the goals 
and initiatives across the enterprise's processes. It allows an emphasis on repeatability, 
consistency and robust process execution across the enterprise. 

However, process assessment (looking at the processes used in the IT organization) may 
be a useful indicator of IT effectiveness but it is not as good an indicator of the satisfaction and 
*|0 vahie as perceived by the users of the IT services (the external customers who are using the IT 

services being provided by the IT organization). 
3l Thus, the prior art systems for assessing inftmnation twAnology delivery systems have 

undesirable disadvantages and limitations which impact the effectiveness of the assessments and 
limit the scope of recommendations for improvements. 



1 5 Summary of the Invention 

The present invention overcomes the disadvantages and limitations of the prior art 
systems by providing a simple, yet effective, way of assessing the effectiveness of an information 
technology organization and for making suggestions for changes to improve the effectiveness of 
the IT services in the future. 
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The present invention uses a matrix of key factors and levels of satisfaction to determine 
where the current information technology organization is and where the customers would like it 
to be in the future. This matrix may be in the form of a generic matrix for all service assessments 
and allow for a quick and relatively simple assessment of the IT service. 
5 The present invention also includes the option of a detailed set of questions to enable a 

detailed analysis of each of several service characteristics and attributes, allowing for a more in 
depth assessment of the IT service. 

The present invention also includes detailed service assessment matrices including one 
for each service characteristic, with a row for each service attributes. These detailed assessment 
f tl 0 matrices allow for an even more detailed analysis of the IT service and providing of the most 
y 1 detailed and intensive analysis. 

Since the tools of the present invention are not mutually exclusive, they allow for the 
I i mixing and matching. Under some circumstances, some of the tools may be used for parts of the 
O analysis (for example, detailed matrices) for intensive analysis couple with a set of questions for 
1 5 an analysis of other factors which is less intense. 

The present invention has the advantage that it focuses on the perception of the 
customers in evaluating the effectiveness of the IT delivery system and is therefor sometimes 
characterized as an evaluation of the delivery or outward manifestation of the IT system rather 
than an inward evaluation of the production system. 

20 The present invention has the advantage that it allows for recommendations to be made 

for improving the perceived effectiveiKss of an IT system. That is, based on the users' 
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perceptions of the IT service, recommendations can be proposed for improving the service and 
increasing the perceived effectiveness of the IT delivery system, 

The present invention has the advantage that, once the level of maturity has been 
identified for various service attributes, improvements to the service regarding one or more 
5 attributes can be proposed. A fbrther advantage of the present invention is that it allov^s a 
clear understanding and communication of who the customers are and how effectively value (as 
perceived by the customers) is being delivered to them. 

The present invention also has the advantage that the analyses promote alignment of the 
f I IT services v^th the business. It provides an insight in to the "why" of activities and provides for 
CflO a base of agreement between the IT service supplier and the recipient of IT services, also 
^ ^ referred to as the customer, 

I;:; The present invention avoids the use of an inward or f»*oduction focus, where the 

f -^ activities of the IT service provider are the chief focus of the analysis. 

lij The present invention is a system and method for three related approaches to assessing 

CI 5 the maturity of the IT services provided to a customer - (1) a ^neric assessment matrix which 

can be used to perform a quick and relatively simple analysis of the IT service; (2) detailed 

questions and directions for each service characteristic and attribute treated in this technique; 

and (3) detailed assessment matrices including a matrix for each of the service characteristics. 

The detailed questions of (2) provide a system which contains questions containing guidance and 

20 background for the context of the assessment matrices to assist in explaining the context and 

target of the various assessment attributes. The questions can be used in interviews and result in 

a slide show used during customer workshops to perform a medium depth assessment of the IT 

service delivery. The detailed assessment matrices outline different levels of implementation of 
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each of the service attributes (five levels in the representative example) and the matrices can be 
used during interviews or workshops or even as a general education on IT services. 

Other objects and advantages of the present invention will be apparent to those skilled in 
the relevant art in view of the following description of the preferred embodiment, taken together 
5 with the accompanying drawings and the appended claims. 

Brief Description of the Drawings 

Having thus set forth some of the limitations and disadvantages of the prior art and some 
^ objects and advantages of the present invention, other objects and advantages will be apparent to 
f II those skilled in the relevant art in view of the following description of the dravdngs illustrating 
jdO the present invention of an improved routing system and method in ^^diich: 
I., Fig. 1 is a block diagram illustration the distinction between IT services and IT processes 

which is usefiil in understandii^ the present in:^ntion and its environment; 
C J Fig. 2 is a matrix of different engagement approaches with different type of assessment 

techniques to illustrate different approaches to analysis of IT services; 
15 Fig. 3 is a matrix of different service attributes versus different level of customer 

satisfaction, as an example of a general IT service maturity assessment matrix; 
Fig. 4 is a sample of service assessment questions for a service attribute; 
Fig. 5 is a sample of an IT service maturity ^sessment matrix; and 
Fig. 6 is a logic flow chart of the process used to assess the IT service (or a component 
20 thereof) using the present invention. 
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Detailed Description of the Preferred Embodimeiit 

In the following description of the preferred embodiment, the best implementations of 
practicing the invention presently known to the inventors will be described with some 
particularity. However, this description is intended as a broad, general teaching of the concepts 
of the present invention in a specific embodiment but is not intended to be limiting the present 
invention to that as shown in this embodiment, especially since those skilled in the relevant art 
will recognize many variations and changes to the specific structure and operation shovm and 
described with respect to these figures. 

Fig. 1 shows a process chart (for a single, yet representative process in a business 
activity) in an effort to illustrate the difference between and IT service and an IF process. An IT 
service may be defined as a specific IT fijnction or output which provides customer value. It is a 
measurable "product which is the basis for doing business with a customer and is delivered 
through a series of implemented business processes and/or activities. An example of an IT 
service is illustrated as "response to queries" identified as 102 in Fig. 1. 

An IT Process is a collection of related activities that take inputs, transforms them, and 
produced outputs that support an enterprise goal. Implemented processes are enable through 
people, tools, and information. An example of an IT process (related to call management in the 
customer service or CSA fimction) is identified in Fig. 1 by the ring 104 which includes a block 
106 where the call is received and logged, a block 108 where the call is analyzed, a block 1 10 
where a call ticket is assigne4 block 1 12 where the query is answered, a block 1 14 where the 
open call tickets are monitored and a block 1 16 where the call is closed. The remainder of Fig. 1 
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illustrates other levels of service and processes and identify the tools, both manual tools and 
automated tools which are used in the performance of the processes. 

It would be useful at this time to provide a definition of two other terms which are used 
throughout this document: An activity is a specific collection of tesks organized around a 
commonly understood result, typically executed in a prescribed sequence. A service flow is a 
description of all the activities, in sequence, required to deliver a service. It includes the 
interprocess flows. 

Fig. 2 is a chart which illustrates different engagement approaches and the use of 
diflTerent tools in completing an assessment Across the top of the chart are the different tools 
available for use in an assessment and down are different types of assessments which can be 
performed. The different types of tools include (but are certainly not limited to) a generic matrix 
identified as column 202, simple questions in column 204, detailed questions in column 206 and 
detailed matrices in colunm 208. Several engagement approaches or types of analyses are listed 
going down the page: a quick, general assessment in line 210, a medium depth assessment in line 
212, an in-4epth, interview-based assessment in line 214 and an in-depth, workshop-based 
assessment in line 216. The respective assessment components which might be chosen for each 
of these types of engagement approaches is illustrated by a check mark in the intersection of the 
column and line (or row) in the chart of Fig. 2. 

Fig. 3 is a simple matrix showing diflFerent levels of IT service maturity based on key 
attributes, with five categories being shown in this matrix. Five columns are shown in levels of 
increasing maturity, from ad hoc in column 302 and repeatable in column 304 to consistent in 
column 306, exceptional in column 308 and world class in column 310, Going down the page, a 
first line or row 312 deals with policies and procedures, a second row 3 14 deals with team work, 
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a third row 316 deals with consistency of delivery, a fourth row 3 1 8 deals with defect handling 
and a fifth row 320 deals with use of results. For each row, an example of each level of maturity 
is provided for each attribute, as a result of assessing the maturity, an IT organization may have 
different levels of maturity for different attributes. The difference may result from different 
priorities or differences in implementation, but, in general, the goal is to reach a higher level of 
maturity for each attribute and to improve on those attributes where the results are relatively less 
mature. So, an organization which has exceptional results in most categories may wish to focus 
on those where the results are merely consistent while an organization which is at the repeatable 
level may have an attribute in which it exhibits "consistent" results and would be more 
concerned about other areas. Obviously as the organization moves to the right in Fig. 3 it is 
increasing customer satisfaction, moving from meeting customer expectations in the middle to 
exceeding customer expectations at the right (and falling short of customer expectations to the 
left) - all at a price of course, and some organizations and some customers are not willing to pay 
for world class services, at least in some areas. 

The number of key attributes in a matrix of the type used in an assessment using the 
system of Fig. 3 may have between ten and twenty attributes, although a lesser number or a 
greater number could be used to advantage. One focus in this process is to determine whether the 
provider and the customer have a common understanding of the scope and objective of the IT 
service being provided and to identify any disparity between the expectations. 

Fig. 4 illustotes sample set of detailed assessment questions to provide a more detailed 
insight into the various attributes than is provided by the matrix of Fig. 3. In this case, the 
deteiled assessment questions are provided for only one service attribute, in this case "definition 
and understanding of the service", as an illustrative example of a part of the set of questions. A 
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similar set of questions would be provided for other service attributes (and there may be many 
such service attributes). In each case, there is a focus area (column 404) associated with each 
attribute (listed in column 402) along with a set of descriptions and considerations in column 
406 and an example in column 408, For a single service attribute there will usually be several 
focus areas, each set forth in a line in Fig. 4, such as lines 410, 412, 414, 416 and 418. Thus, in 
line 414, the focus area is whether the scope of service is defined and the considerations in 
column 406 include where the service inclusions and exclusions are clear, agreed to by the 
parties and documented The answers to these questions will lead the assessment to a 
determination of the level of maturity as well as identifying areas for improvement. For example, 
if there is a disagreement on the scope of the service between the provider and the customer, it 
will be hard to have a mature service and the remedy suggested may be that the provider and the 
customer agree on the scope of the service and document it One key aspect to assessing a 
service is to note the gap between the reality and the perceptions as seen by the customer and by 
the provider. 

Fig, 5 is a third example of an assessment approach where detailed matrices are provided 
for assessing the level of maturity of a service. Fig. 5 is similar to Fig. 3 in providing different 
levels of maturity across the horizontal columns and a service attribute in the vertically-arranged 
rows but Fig. 5 is meant to provide a more detailed set of attributes and allow for an 
identification of the current level of service (labeled as "now" in column 510) as well as a 
desired level of service in column 520, (labeled as "goal"), A comparison of the "now" results 
with the "goal" for each service attribute can lead quite easily to conclusions about areas for 
improvement (where the now level of service is less than the goal level of service) as well as 
areas for possible economy (where the now level of service in column 510 exceeds the goal level 
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of service in column 520). In short, the IT organization should be using resources in areas which 
are important to the business objectives of the organization and not on achieving results without 
regard to the priorities of the organization. In this manner, the objectives of the IT organization 
in delivering its services may become better aligned with the objective of the organization as a 
whole. 

Fig. 6 is a flow chart illustrating the logical flow of the information technology service 
assessment of the present invention using the elements and tools described above to provide an 
analysis of the current state of IT service and recommendations for improving the service. 
Starting at block 602, the consultant and the customer identify the service or services which are 
to be assessed. This may be the entire IT operation with all of its many services or a selected 
subset of those services, based on the time and money available for study or a perception of 
things which are in greater need of attention. For example, an IT service may provide data 
processing applications for invention control, financial reporting, payroll and computer aided 
design and manufacturing (CADAM), among other applications. If the cost in time and effort is 
too great to analyze all of the foregoing, it may be determined that inventory control has the 
greatest potential for service improvements and other benefits to the business and that service 
may be analyzed first 

Next, at block 604, the consultant determines the assessment technique(s) to be used. 
This may be a generic matrix as presented and discussed in connection with Fig. 3, detailed 
questions as presented and discussed in connection with Fig. 4, detailed matrices as discussed in 
connection with Fig. 5 or some combination of the techniques. Thus, in an example, the 
consultant has two formats (matrices and questions) available and each is available in a simple 
form and in a detailed form, although in practice, multiple levels could be developed between 
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the simple and the detail to accommodate any desired level of analysis. At block 606 the 
assessment tools are reviewed and appropriate tools are selected for the assessment Next, the 
service attributes are selected at block 608 and modified at block 610 to fit the circumstances. 
Depending on the type of service being assessed and the level of that assessment, it may not be 
necessary to ask all the questions for an attribute or to evaluate all the attributes of each service. 
For example, services such as "ensuring availability of servers" may not need to focus on a 
human interaction as much as a services which relies on people for the delivery of that service. 

At block 612 the data sources are determined and at block 6 14 the services are assessed 
using the selected technique - either the matrix or questions. It is essential to understand the 
views of the provider and the customer in doing the assessment, but the inforaiation may be 
captured through various techniques. Where workshops are the preferred means of gathering 
information, then matrices may be more suitable. Where customer viewpoint's are sought by 
questions, it may be necessary to reword the questions to make them understandable by the 
customer - preferably in language which is easily understood by the customer being interviewed 

Following the gathering of information in the assessment of block 614, the facts and 
findings are consolidated at block 616, recommendations and conclusions are set out in block 
6 1 8 and the results are reported at block 620, 

Of course, many modifications of the present invention will be apparent to those skilled 
in the relevant art in view of the foregoing description of the preferred embodiment, taken 
together with the accompanying drawings. For example, the present invention is not limited to 
the types of analysis tools which are described above and other tools may be used in addition to 
or in place of the matrices and questions disclosed For example, a customer may maintain a log 
which provides useful information regarding the customer complaints and praise for the IT 
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services and this log may provide some useful information and avoid having to ask questions 
about the state of the IT service as perceived by the users. The levels of service may also be an 
indicator of the satisfaction or dissatisfaction with the IT services. Some of the features and 
elements of the present invention may be useful without the corresponding use of other features 
and elements. Additionally, many modifications can be made to the system implementation and 
the method of presenting conclusions and recommendations. Accordingly, the foregoing 
description of the preferred embodiment should be considered as merely illustrative of the 
principles of the present invention and not in limitation thereof 
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